{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "aa065a5a",
   "metadata": {},
   "source": [
    "* 时间：2023-10-11  \n",
    "* Web数据挖掘  \n",
    "* week06"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e5cff33",
   "metadata": {},
   "source": [
    "# 本周内容（动态加载Ajax）数据获取"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c72dace6",
   "metadata": {},
   "source": [
    "## 实践项目：liepin数据抓取"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f51eb97",
   "metadata": {},
   "source": [
    "* 网站：liepin.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14e5f946",
   "metadata": {},
   "source": [
    "* 尝试 requsets--html 获取页面数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "13ed81d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from requests_html import HTMLSession\n",
    "session = HTMLSession()\n",
    "\n",
    "r = session.get('https://www.liepin.com/zhaopin/?city=050090&dq=050090&pubTime=&currentPage=0&pageSize=40&key=%E4%BA%A7%E5%93%81%E7%BB%8F%E7%90%86&suggestTag=&workYearCode=1&compId=&compName=&compTag=&industry=&salary=&jobKind=&compScale=&compKind=&compStage=&eduLevel=&otherCity=&sfrom=search_job_pc&skId=3yz0xxc2fcm8u84ttah7coo1zrrfcsma&fkId=3yz0xxc2fcm8u84ttah7coo1zrrfcsma&ckId=3yz0xxc2fcm8u84ttah7coo1zrrfcsma&scene=condition&suggestId=')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5b3d7baf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag(\\'js\\', new Date()); gtag(\\'config\\', \\'G-54YTJKWN86\\');\\n【深圳招聘信息_深圳人才网招聘信息】-猎聘\\n全国\\n首页\\n职位\\n校园\\n海归\\n社区\\n简历优化\\n猎聘APP\\n我是猎头\\n我是招聘方\\nNEW\\n登录/注册\\n热门职位\\n推荐职位\\n热门行业\\n周边城市\\n热门城市\\n其他公司\\n相关公司\\n手机版\\n深圳IC电子招聘深圳ATM选址经理招聘深圳BO报表开发工程师招聘深圳产品结构分析招聘深圳会计稽核主管招聘深圳信托投资招聘深圳变频设计师招聘深圳厨房行政主管招聘深圳地产项目副总经理招聘深圳成本会计核算招聘深圳意大利网站客服招聘深圳工艺设计主管招聘深圳排版技术总监招聘深圳发酵工段长招聘深圳学校招生助理招聘深圳地产渠道拓展招聘深圳外滩豪宅销售招聘深圳韩语社群运营招聘深圳驻厂外派督导招聘深圳供应商活动运营招聘深圳云平台存储专家招聘深圳公募基金产品总监招聘深圳初中双语物理老师招聘深圳冲压生产工段长招聘深圳投资部门管培生招聘深圳数据链系统工程师招聘深圳海外口碑运营专员招聘深圳生物药注册经理招聘深圳综合文员岗招聘深圳行销训练岗招聘\\n深圳APP推广招聘深圳期货客户经理招聘深圳互联网金融产品经理招聘深圳计量工程师招聘深圳FAE现场应用工程师招聘深圳商务专员/助理招聘深圳媒介投放招聘深圳助教招聘深圳通信设备工程师招聘深圳游戏策划师招聘深圳焊工招聘深圳SEO招聘深圳环境评价工程师招聘深圳通信测试工程师招聘深圳集装箱业务招聘深圳会展活动策划招聘深圳机械结构工程师招聘深圳再保险招聘深圳原画师招聘深圳银行综合业务专员/助理招聘深圳列车长/司机招聘深圳法务总监招聘深圳环境/健康/安全管理招聘深圳计量工程师招聘深圳生产项目管理招聘深圳无线网络优化招聘深圳其他语种翻译招聘深圳汽车电子/电器工程师招聘深圳汽车定损/车险理赔招聘深圳酒店前台招聘\\n深圳日化招聘深圳担保/拍卖/典当招聘深圳广播/影视/录音招聘深圳广告/公关/会展招聘深圳计算机软件招聘深圳室内娱乐招聘深圳其他商务服务业招聘深圳政府/公共事业招聘深圳新零售招聘深圳日用杂品招聘深圳医药外包招聘深圳其他教育培训招聘深圳工程管理/勘察/监理招聘深圳新闻和出版业招聘深圳仪器仪表招聘深圳办公用品/设备招聘深圳汽车零部件及配件招聘深圳智能硬件招聘深圳检测/认证招聘深圳新能源汽车招聘深圳汽车交易/后市场招聘深圳家具/家居招聘深圳人工智能招聘深圳体育招聘深圳房地产租赁/中介招聘深圳电气机械/器材招聘深圳生物技术招聘深圳航空/航天设备招聘深圳珠宝/首饰招聘深圳文娱用品/器材招聘\\n广州招聘惠州招聘梅州招聘汕尾招聘河源招聘阳江招聘清远招聘东莞招聘韶关招聘中山招聘珠海招聘潮州招聘揭阳招聘汕头招聘云浮招聘佛山招聘江门招聘湛江招聘茂名招聘肇庆招聘\\n北京招聘网上海招聘网广州招聘网厦门招聘网杭州招聘网郑州招聘网南京招聘网天津招聘网重庆招聘网成都招聘网苏州招聘网商丘招聘网大连招聘网济南招聘网宁波招聘网无锡招聘网青岛招聘网沈阳招聘网台州招聘网西安招聘网武汉招聘网\\n深圳市联众时空网络有限公司深圳市鼎永亿投资有限公司深圳市盛世金融控股集团有限公司深圳市昇隆启泰资产管理有限公司深圳宇泰通电子技术有限公司深圳郎琴新媒体有限公司深圳前海亿泽新能源发展有限公司深圳市闻扬实业有限公司海岳文化传播(深圳)有限公司深圳市汉盛堂科技有限公司深圳永清爱能森新能源工程技术有限公司深圳市前海金潮达科技有限公司深圳市中恒花旗投资有限公司深圳市前海亿联科技有限公司深圳市启越教育文化发展有限公司深圳市衣缘巧依服饰有限公司深圳市研创精密设备有限公司深圳市国仁光电有限公司深圳市前海融金通互联网金融服务有限责任公司深圳市幸福指数金融服务有限公司深圳前海华利投资管理有限公司深圳市华乐兄弟广告有限公司深圳市信源兴模具有限公司深圳市前海一心德实业发展有限公司深圳美城家园能源投资有限公司国采晟信有限公司深圳市腾沣建筑材料有限公司深圳市进远科技有限公司深圳快培教育有限公司深圳市怡音科技有限公司\\n深圳市全民基金管理有限公司深圳鸿跃文化传媒有限公司深圳市启高启新科技有限公司深圳市金九天视实业有限公司深圳美雅因私出入境服务有限公司深圳迪佰特科技有限公司深圳市普朗科技有限公司深圳市智汇运筹网络科技有限公司深圳市来格科技有限公司固胜智能FRAME FAN吴越服饰深圳市多多财务代理有限公司光大节能照明深圳市鑫涔科技有限公司深圳市天勤创展机电设备有限公司小鹿商业深圳市钧强科技有限公司深圳酷享信息科技有限公司深圳市励研科技有限公司\\n深圳招聘\\n当前位置：\\n深圳招聘网 > 深圳招聘信息\\nwindow.addEventListener(\\'message\\', function(event) {if (event.data && event.data.type === \\'footer-height\\') {document.querySelector(\\'#common-footer\\').setAttribute(\\'height\\', event.data.height + \\'px\\') }}, false) (function () { var imEntryScript = \\'<script crossOrigin=\"anonymous\" src=\"//concat.lietou-static.com/fe-imc-pc/v6/latest/entry.js?q=\\'.concat( Math.floor(Date.now() / 180000), \\'\"><\\\\/script>\\' ); document.write(imEntryScript); })(); { \"@context\": \"https://ziyuan.baidu.com/contexts/cambrian.jsonld\", \"appid\": \"1586030202028057\", \"@id\": \"https://www.liepin.com/city-sz/zhaopin/?scene=seo\", \"title\": \"【深圳招聘信息_深圳人才网招聘信息】-猎聘\", \"description\": \"猎聘深圳招聘信息提供真实的深圳人才招聘信息，深圳招聘信息有超过10000多高薪人才招聘信息职位任你选，找深圳招聘信息就来深圳人才网招聘信息频道。\", \"upDate\": \"2023-11-29T08:00:00\", \"data\": { \"WebPage\": { \"headline\": \"【深圳招聘信息_深圳人才网招聘信息】-猎聘\", \"pcUrl\": \"https://www.liepin.com/city-sz/zhaopin/?scene=seo\", \"wapUrl\": \"https://m.liepin.com/city-sz/zhaopin/?scene=seo\" } } } var _hmt = _hmt || []; (function () { var hm = document.createElement(\"script\"); hm.src = \"//hm.baidu.com/hm.js?a2647413544f5a04f00da7eee0d5e200\"; var s = document.getElementsByTagName(\"script\")[0]; s.parentNode.insertBefore(hm, s); })(); (function () { var bp = document.createElement(\\'script\\'); var curProtocol = window.location.protocol.split(\\':\\')[0]; if (curProtocol === \\'https\\') { bp.src = \\'https://zz.bdstatic.com/linksubmit/push.js\\'; } else { bp.src = \\'http://push.zhanzhang.baidu.com/push.js\\'; } var s = document.getElementsByTagName(\"script\")[0]; s.parentNode.insertBefore(bp, s); })(); ;(function (n, e, r, t, a, o, s, i, c, l, f, m, p, u) { o = \\'precollect\\' s = \\'getAttribute\\' i = \\'addEventListener\\' c = \\'PerformanceObserver\\' l = function (e) { f = [].slice.call(arguments) f.push(Date.now(), location.href) ;(e == o ? l.p.a : l.q).push(f) } l.q = [] l.p = { a: [] } n[a] = l m = document.createElement(\\'script\\') m.src = r + \\'?aid=\\' + t + \\'&globalName=\\' + a m.crossorigin = \\'anonymous\\' e.getElementsByTagName(\\'head\\')[0].appendChild(m) if (i in n) { l.pcErr = function (e) { e = e || n.event p = e.target || e.srcElement if (p instanceof Element || p instanceof HTMLElement) { n[a](o, \\'st\\', { tagName: p.tagName, url: p[s](\\'href\\') || p[s](\\'src\\') }) } else { n[a](o, \\'err\\', e.error || e.message) } } l.pcRej = function (e) { e = e || n.event n[a](o, \\'err\\', e.reason || (e.detail && e.detail.reason)) } n[i](\\'error\\', l.pcErr, true) n[i](\\'unhandledrejection\\', l.pcRej, true) } if (\\'PerformanceLongTaskTiming\\' in n) { u = l.pp = { entries: [] } u.observer = new PerformanceObserver(function (e) { u.entries = u.entries.concat(e.getEntries()) }) u.observer.observe({ entryTypes: [\\'longtask\\', \\'largest-contentful-paint\\', \\'layout-shift\\'] }) } })( window, document, \\'https://concat.lietou-static.com/fe-lib-pc/v6/apmplus/1.7.0/browser.cn.js\\', 0, \\'apmPlus\\' ) window.apmPlus(\\'init\\', { aid: 460715, token: \\'26ca6df079bf44f09bd002af5fdb382c\\', plugins: { ajax: false, fetch: false, pageview: true, resource: false, performance: { longtask: true } }, sample: { rules: { pageview: { enable: true, sample_rate: 0.01 }, performance: { sample_rate: 0.0005 }, performance_timing: { sample_rate: 0.0005 }, performance_longtask: { sample_rate: 0.0005 } } } }) window.apmPlus(\\'start\\')'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.html.text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fcffed5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "r = session.post('https://api-c.liepin.com/api/com.liepin.searchfront4c.pc-search-job')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2e69d21b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fb260c69",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'flag': 0, 'code': '-1400', 'msg': '出错了（400）！'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "36f4ee14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko) Version/10.1.2 Safari/603.3.8', 'Accept-Encoding': 'gzip, deflate, br', 'Accept': '*/*', 'Connection': 'keep-alive', 'Cookie': 'XSRF-TOKEN=JwjXIHoDQPCi52otwJqcgA; __gc_id=72d6d938f8a542f9827d94353dc44185', 'Content-Length': '0'}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.request.headers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a29c7e4f",
   "metadata": {},
   "source": [
    "* 无法请求到数据的原因"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "319b3086",
   "metadata": {},
   "source": [
    "> 1. 模拟真实用户代理  \n",
    "> 2. 提供请求的参数（类比API）  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e0288cdf",
   "metadata": {},
   "outputs": [],
   "source": [
    "liepin_headers = {\n",
    "    \"Accept\":\"application/json, text/plain, */*\",\n",
    "    \"Accept-Encoding\":\"gzip, deflate, br\",\n",
    "    \"Accept-Language\":\"zh-CN,zh;q=0.9\",\n",
    "    \"Connection\":\"keep-alive\",\n",
    "    \"Content-Length\":\"487\",\n",
    "    \"Content-Type\":\"application/json;charset=UTF-8;\",\n",
    "    \"Cookie\":\"inited_user=464b5299797bde79a6aeac4e54e72749; __gc_id=fcb411cd19404d6c8519f9289c795f0a; _ga=GA1.1.737936061.1681902650; __uuid=1681902669648.13; need_bind_tel=false; c_flag=92fc82dab3c60b517e05029c26bb865c; imClientId=b5e1273d5cb093f4b435a5bdfa4014a6; imId=b5e1273d5cb093f407ef590fee2a579a; imClientId_0=b5e1273d5cb093f4b435a5bdfa4014a6; imId_0=b5e1273d5cb093f407ef590fee2a579a; new_user=false; XSRF-TOKEN=dh0XXpODSwCKaWPF_5TMDQ; __tlog=1697022022074.43%7C00000000%7C00000000%7Cs_o_007%7Cs_o_007; Hm_lvt_a2647413544f5a04f00da7eee0d5e200=1697022022; hpo_role-sec_project=sec_project_liepin; hpo_sec_tenant=0; UniqueKey=cb5430ca53f7a1c16bc56763ae71c6f2; liepin_login_valid=0; lt_auth=uu0La3ECnA7%2F5nba2zFbta1O2o%2BsAzrM8S4LgUgC1YW6XqXg4P%2FmQAOHqrEG%2BCoIqxshIfUzMLb3Muj8y3JK7UMS%2BVGnlZ6utf6k0HsDUeVtIMW2vezHg%2FXUQp0lkkAA8nJbpEIL%2BVzO; user_roles=0; user_photo=5f8fa3baf6d1ab58476f324908u.png; user_name=%E5%90%B4%E8%8C%B5%E8%8C%B5; inited_user=464b5299797bde79a6aeac4e54e72749; imApp_0=1; _ga_54YTJKWN86=GS1.1.1697022021.9.1.1697023068.0.0.0; __session_seq=12; __uv_seq=12; Hm_lpvt_a2647413544f5a04f00da7eee0d5e200=1697023069; fe_im_opened_pages=_1697023070436; fe_im_socketSequence_new_0=5_5_5; fe_im_connectJson_0=%7B%220_cb5430ca53f7a1c16bc56763ae71c6f2%22%3A%7B%22socketConnect%22%3A%224%22%2C%22connectDomain%22%3A%22liepin.com%22%7D%7D; __tlg_event_seq=60\",\n",
    "    \"Host\":\"api-c.liepin.com\",\n",
    "    \"Origin\":\"https://www.liepin.com\",\n",
    "    \"Referer\":\"https://www.liepin.com/\",\n",
    "    \"Sec-Ch-Ua\":\"\\Google Chrome\\\";v=\\\"119\\\", \\\"Chromium\\\";v=\\\"119\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\"\",\n",
    "    \"Sec-Ch-Ua-Mobile\":\"?0\",\n",
    "    \"Sec-Ch-Ua-Platform\":\"\\Windows\\\"\",\n",
    "    \"Sec-Fetch-Dest\":\"empty\",\n",
    "    \"Sec-Fetch-Mode\":\"cors\",\n",
    "    \"Sec-Fetch-Site\":\"same-site\",\n",
    "    \"User-Agent\":\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36\",\n",
    "    \"X-Client-Type\":\"web\",\n",
    "    \"X-Fscp-Bi-Stat\":\"{\\\"location\\\": \\\"https://www.liepin.com/zhaopin/?currentPage=0&pageSize=40&city=050090&dq=050090&pubTime=&key=%E4%BA%A7%E5%93%81%E7%BB%8F%E7%90%86&suggestTag=&otherCity=&industry=&scene=condition&skId=n6tbyab50hqfp57f2938dc8nxy9k5btb&fkId=n6tbyab50hqfp57f2938dc8nxy9k5btb&ckId=n6tbyab50hqfp57f2938dc8nxy9k5btb&sfrom=search_job_pc&suggestId=\\\"}\",\n",
    "    \"X-Fscp-Fe-Version\":None,\n",
    "    \"X-Fscp-Std-Info\":\"{\\\"client_id\\\": \\\"40108\\\"}\",\n",
    "    \"X-Fscp-Trace-Id\":\"41aafc33-4b97-4094-a7bf-c0858337bf9f\",\n",
    "    \"X-Fscp-Version\":\"1.1\",\n",
    "    \"X-Requested-With\":\"XMLHttpRequest\",\n",
    "    \"X-Xsrf-Token\":\"AC-8dG1sTSqXIEek6c65WQ\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "00c3ab79",
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "EOL while scanning string literal (1087435623.py, line 3)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  Input \u001b[1;32mIn [8]\u001b[1;36m\u001b[0m\n\u001b[1;33m    \"job: {\u001b[0m\n\u001b[1;37m           ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m EOL while scanning string literal\n"
     ]
    }
   ],
   "source": [
    "payload = {\n",
    "  \"data\":{\n",
    "  \"job: {\n",
    "   \"advViewFlag\": \"\",\n",
    "\"dataPromId\": \"\",\n",
    "\"dq\": \"\",\n",
    "\"h5OuterLink\": \"\",\n",
    "\"jobId\": \"\",\n",
    "\"jobKind\": \"2\",\n",
    "\"labels\": \"\", \n",
    "\"link\": \"\",\n",
    "\"pcOuterLink\": \"\",\n",
    "\"refreshTime\": \"20210702211502\",\n",
    "\"requireEduLevel\": \"本科\",\n",
    "\"requireWorkYears\": \"经验不限\",\n",
    "\"salary\": \"\",\n",
    "\"title\": \"广告策划\",\n",
    "\"topJob\": \n",
    "false\n",
    "  },\n",
    "  \"passThroughForm\": {\n",
    "    \"ckId\": \"5xohduldzgub86rae516slz9avwwm288\",\n",
    "    \"scene\": \"input\",\n",
    "    \"skId\": \"3yz0xxc2fcm8u84ttah7coo1zrrfcsma\",\n",
    "    \"fkId\": \"8omvjs0ooq0bh7rast7ijuw0edjtn4pb\",\n",
    "    \"sfrom\": \"search_job_pc\"\n",
    "  }\n",
    "}\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49ad3b23",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "r = session.post(url = 'https://api-c.liepin.com/api/com.liepin.searchfront4c.pc-search-job',\n",
    "                 data = json.dumps(payload), ##post请求的参数\n",
    "                 headers = liepin_headers    ##模拟真实用户\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bce9b9b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "liepin_data = r.json()\n",
    "liepin_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0aba7ddf",
   "metadata": {},
   "outputs": [],
   "source": [
    "r.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "865aaabb",
   "metadata": {},
   "source": [
    "## 动态页面获取关键步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9313187e",
   "metadata": {},
   "source": [
    "> 1. 如果获取不到数据，请首先做模拟用户，即模拟浏览器headers  \n",
    "> 2. 检查F12 Network部分XHR中参数的信息，确保与浏览器参数一致，先保证可以获取到数据  \n",
    "> 3. 在2的基础上，根据我们的数据需求优化参数数据  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8374fb20",
   "metadata": {},
   "source": [
    "* 关于liepin核心参数的详细信息获取（帮助我们后续的分类查询）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e71f9e80",
   "metadata": {},
   "outputs": [],
   "source": [
    "r = session.post(\n",
    "            url = 'https://api-c.liepin.com/api/com.liepin.searchfront4c.pc-search-job-cond-init',\n",
    "            data = json.dumps(payload), # POST请求带的参数\n",
    "            headers = liepin_headers    # 模拟真实用户\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12471aa2",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "condition = r.json()\n",
    "condition"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "990267d6",
   "metadata": {},
   "source": [
    "* 根据数据需求优化请求参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66287e89",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06b11ff5",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.json_normalize(liepin_data['data']['data']['jobCardList'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e4502a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "compScales  = pd.json_normalize(condition['data']['compScales'])\n",
    "compScales"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53183dce",
   "metadata": {},
   "outputs": [],
   "source": [
    "compScales  = pd.json_normalize(condition['data']['compScales'])\n",
    "compScales"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77a41d03",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36772a50",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7af72d93",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61379f6b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89c426c4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf2f3e4a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
